FAFNet: Fully aligned fusion network for RGBD semantic segmentation based on hierarchical semantic flows

Depth maps are acquirable and irreplaceable geometric information that significantly enhances traditional color images. RGB (RGBD) images have been widely used in various image analysis applications, but they still very limited due to challenges from different modalities misalignment between depth. In this paper, a Fully Aligned Fusion Network (FAFNet) for RGBD semantic segmentation is presented. To improve cross-modality fusion, new fusion block proposed, features depth first fused by an attention cross module then aligned flow. A multi-layer structure also designed hierarchically utilize the block, which not only eases issues of low resolution noises reduces loss upsampling process. Quantitative qualitative evaluations on both NYU-Depth V2 SUN RGB-D dataset demonstrate FAFNet model outperforms state-of-the-art methods.